1. Field of the Invention
The present invention relates to an image template matching method which is used when an object in a motion picture is tracked or when binocular stereopsis for reproducing a shape of an object from a plurality of images is performed, and an image processing device.
2. Description of the Related Art
A template matching is a technology for comparing a certain image with a template set in another image to obtain matching, and it is quite often utilized as a fundamental method for finding out points or areas corresponding to each other among a plurality of images. In order to track an object in a motion picture, an area analogically closest to a template, that is, an area having the highest correlation with the template (matching area) is searched from the following other frame images using an area of an object in an initial frame image of the motion picture as a template. Then, it is determined that the object has been moved to the matching area. On the other hand, in a binocular stereopsis, feature points corresponding to each other among a plurality of static images picked up from two or more different positions are found using one of the images as a template according to the template matching. A three dimensional shape of an object which has been picked up is calculated from positional information about a camera and information about position deviations among the corresponding feature points on the respective static images.
Regarding an object tracking in the motion pictures, when the entire object is used as a template, the precision of the template matching is lowered by the influence of the deformation of the object. Therefore, such a process should be employed that an appropriate number of tracking points are set within the object area, a template matching is performed using templates having an appropriate size containing the tracking points. A place to which the object is moved is determined from these template matched positions. In this case, it is desirable that the number of tracking points (the number of templates) is lessened as few as possible in order to shorten a calculating time.
In order to perform the tracking of an object in a short time with a high precision, a tracking point to which the template matching can be performed with a high precision must be selected. Similarly, also in the binocular stereopsis, the three-dimensional shape cannot be precisely calculated unless feature points to which the template matching can be precisely performed has been previously selected.
Thus, how to select tracking points or feature points is an important problem which has an influence on the performance. Hereinafter, the tracking point used for tracking an object in motion pictures and the feature point used for binocular stereopsis will be treated synonymously, and both are generically referred to as reference point.
Conventionally, as such a reference point, namely, a tracking point for tracking an object and a feature point for binocular stereopsis, a point where the variance of pixel values of surrounding pixels is large, a corner point, a point whose local curvature is large (see Reference 1: Toshimitsu Kaneko and Osamu Hori, “Object Tracking Method with Affine Deformation Estimation Using Robust Statistics”, The fifth image sensing symposium, C-18, pp. 129–134, June, 1999) and the like are used. Moreover, a feature point suitable for gradient method by which an optical flow is found, which was proposed in Reference 2: Carlo Tomasi and Takeo Kanade, “Shape and Motion from Image Streams: a Factorization Method-part 3, Detection and Tracking of Point Features,” CMU-CS-91-132, Carnegie Mellon University, 1991, has been also utilized.
However, since a method of selecting a point of which the dispersion of the values of surrounding pixels described above is large, a corner point, a point whose local curvature is large or the like as a reference point is based on the standard considered originally from the human intuition, it is not guaranteed that an appropriate reference point is selected. Particularly, since a method of making a point whose local curvature is large as a reference point is considered only for the portions of motion picture close to the reference point, it cannot determine that it is not appropriate as a reference point in the case where a similar pattern is located slightly apart from the reference point. Furthermore, since the method described in the Reference 2 is a method of selecting a feature point specialized for the gradient method, it cannot necessarily select an appropriate feature point as a reference point used for template matching.
On the other hand, also regarding the other parameters such as a size of template used for template matching, a resolution when the template matching is performed and the like except for a reference point, conventionally, since these are determined depending upon the human experiences and intuition, there is a problem that it is not certain whether or not an appropriate value is selected.